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[57] ABSTRACT 

Aphotofinishing system for automatically processing image 
and associated image data pursuant to customer output 
requests. The system includes an order manager operative to 
receive and control processing of the output requests and at 
least one source of image related data corresponding to the 
output request. An input interface receives the image related 
data from the source and converts the data to a digital data 
stream. A memory is included for storing the digital data 
stream. The system further includes a data parser disposed in 
communication with the memory to extract selected data 
streams according to the order manager and to reduce the 
data into respective image files having respective groups of 
data fields. An output module is respoasive to the order 
manager and is operative to produce a photofinished output 
organized with respect to the data fields. 

31 Claims, 24 Drawing Sheets 



.30 



ORDER ^07 
MANAGER 



27 ^ 



FILM 
DEVELOPING 





FILM 




DIGITIZING 



IX 

READER 



29 

H 



BUFFER 



26 




33 

I root- 

35 x 



ESC DATA 
NET 



28 



digitizer] — 

37 



1 ORDER 




i MANAGER i 


31 
1 




BUFFER 





3 n 
34 



PRINT 0PTICALS 
OR 

RE-BAG ROLL 



20 



DIGITAL 
DATA STREAM 
STORAGE 



38 



40 

z _ 

44 



/ 



46^1 



DIGITAL 
DATA FILE 
STORAGE 




AUTOMATIC 
BUILD- IT 
UNIT 





DIGITAL 
OPTICAL PRINT 
WRITER 

PCD/DVD 
WRITER 




1 

AUDIO 




AUDIO 


CASSETTE 




CD 



02/19/2004, EAST Version: 1.4.1 



U.S. Patent 



Nov. 14, 2000 Sheet 1 of 24 



6,147,742 




02/19/2004, EAST Version: 1.4.1 



U.S. Patent 



Nov. 14, 2000 



Sheet 2 of 24 



6,147,742 




LUT 



WRITE 
ELEMENT 
# 


CORR'N* OFFSET 


I 1 


•1 o 


2 


•1 o 


3 


1 0 


4 


.82 0 


5 


1 0 


6 


1 0 


7 


1 0 


n 


.7 +.13 


ni 


1.22 0 


nj 


1 o 


n 


1 +.05 



FIG. 8 



02/19/2004, EAST Version: 1.4.1 



U.S. Patent Nov. 14, 2000 Sheet 3 of 24 



6,147,742 



RECEIVING BATCH REQUESTS 



•50 



TRANSFORMING IMAGE AND AVOID 
INFORMATION INTO DATA STREAM 



ACCUMULATING DATA THROUGH 
INPUT INTERFACE 



-52 



54 



INTERPRETING AND 
CLASSIFYING DATA 



■56 



AUTOMATICALLY GATHERING AND 
CO-PROCESSING CUSTOMER DATA 



ESTABLISHING CORRESPONDENCE 
BETWEEN DIGITAL IMAGES AND 
DIGITAL DATA FILES 



58 



•60 



AUTOMATICALLY SORTING AND 
FORMATTING IMAGE AND AUDIO 
FILE SETS 



•62 



FIG. 2 



02/19/2004, EAST Version: 1.4.1 



U.S. Patent 



Nov. 14, 2000 



Sheet 4 of 24 



6,147, 



(^START^) 




YES 



72 



CATALOG ORDER REQUEST 
FOR WORKLOAD MGMT 



RELAY DATA FILE ID AND 
INPUT PORT ID TO DIGITAL 
DATA STREAM STORAGE 



YES. 



REQUEST FOR 
STATUS UPDATE' 
FROM OTHER PF 
.EQUIPMENTS 



NO 



76 



74 



UPDATE STATUS IN 
ORDER CATALOG 



•78 



QUE/ENABLE NEXT PROCESS 
(MAY NEED TO BE QUED 
FOR WORKLOAD MGMT.) 




80 



FIG. 3 



REMOVE FROM CATALOG 




•84 



STOP 



02/19/2004, EAST Version: 1.4.1 



U.S. Patent Nov. u, 2000 



Sheet 5 of 24 



6,147,742 




98 



ENABLE DATA FLOW 
FROM INPUT PORT 
SPECIFIED 



100 -l 



RECEIVE AND STORE 
DATA STREAM TO FILE 



LOOK-UP THE FILE 
ID AND LOCATE 
FILE REQUESTED 



102 



UPON END-OF-FILE IN STREAM 

1) CLOSE FILE AND 
2) NOTIFY ORDER MANAGER 
THAT STATE=IDLE 



RELAY DATA 
STREAM TO 
PARSER 



T 

108 



FIG. 4 



02/19/2004, EAST Version: 1.4.1 



U.S. Patent 



Nov. 14, 2000 



Sheet 6 of 24 



6,147,742 



LLI 
Q 
O 

O 

s 



<3 < 
CC < Q 



© 



_J ^, LU 

< H 2 

P > < 0C & P 

O < < I— I— UJ 
<CC Q CO CO CO 



© 



CO 

Q < 
O > < 
Q> H 
Z < < 

LU DC Q 



-I \ UJ 3fc 

< h Z UJ 

3> <CC OLfT .2 

o<<HHiu^cr: 
<ccacococoi=LL 




Z CO CO CO 

<Ouj=jo5 



® 



UJ 

LU 
X 



Q 
< 



LU 



LU 

< 

CC 



z 
o 

CO 
CO 
LU 
CC 



CC 



_o 

- uj " cc fr cc 2 < 
oco 00$ o < q 



OaO< 

Q5QDC 

< co < a. 



x 
< 



2 

Q 



< 
o 

CO 

>• 

UJ 
CC 

o 



X 

< 



5 

Q 

QC 
O 

CO 
O 
LU 

< 

o 

CO 

> 

LU 

CC 

o 



< 

Q 



® 



> 



Q CC CO LU 



© 



CO 



CO 
LU 



02/19/2004, EAST Version: 1.4.1 



U.S. Patent 



Nov. 14, 2000 



Sheet 7 of 24 



6,147,742 



120 



START 




BEGIN TO PARSE A 
NEW RAW DATA STREAM 




FIG.6 



CREATE A NEW 
IMAGE DATA FILE 



126 



TAG HEADER 
AS USUAL 




FIG.6A 



TAG FILE HEADER WITH 
ENSUING DATA: CID# OR 
DIGITAL CAMERA ID# 

AUDIO P ARAMS, 
ENCODING PARAMS 



144 



CREATE TEMPORARY FILE 
FOR DATA CALIBRATION 
CONVERSIONS: USE FILE 

CALIBRATION SCHEME ID# 
TO FORMAT LUT'S 



02/19/2004, EAST Version: 1.4.1 



U.S. Patent 



Nov. 14, 2000 Sheet 8 of 24 



6,147,742 



146 



CALCULATE WRITE ELEMENT 
OFFSET, GAIN CORRECTIONS 
FROM TONE SERIES DATA 



148^ 



CALCULATE TONE SCALE 
TO BINARY VALUE CONVERSION 
FROM CORRECTED TONE SERIES DATA 



150 



PARSE/CONVERT RAW DATA 
TO CALIBRATED BINARY 
VALUES & WRITE TO FILE 



END OF 
/AUDIO FOUND' 
(START/STOP 
SENTINEL) 



YES 



FIG.6B 



02/19/2004, EAST Version: 1.4.1 



U.S. Patent 



Nov. 14, 2000 



Sheet 9 of 24 



6,147,742 




02/19/2004, EAST Version: 1.4.1 



U.S. Patent 



Nov. 14, 2000 



Sheet 10 of 24 



6,147, 



DIGITAL 
VALUE 
^n4 



V 



RAW 
A/D 
VALUE 

A-- 



\ 



\ 



\ 



0- 



TT-1 

Drrin 



Dmax. 



co t S $ S 

UJ UJ LU UJ Ld 

^ Z Z Z 2T 

o o p o p 



ro 
UJ 

z 
o 



AVERAGE FILM DENSITY 
(<> n TONES IN PRE-SPECIFIED ORDER) 



'J 



FIG 9 



02/19/2004, EAST Version: 1.4.1 



U.S. Patent Nov. 14, 2000 Sheet 11 of 24 



DETERMINING IMAGE OUTPUT 
PATH REQUESTED 



LOADING RULES 
FOR ORGANIZING 



i 



EXTRACTING MERGE 
ID CODE 



i 



CARRYING OUT 
FILE SEARCH 



COMPILING FILES INTO 
LOOK-UP TABLE 



I 



LINKING IMAGE-AUDIO 
PAIRS 



PASSING CONTENT TO 
AUTO BUILD-IT MODULE 



FIG. 10 



02/19/2004, EAST Version: 1.4.1 



U.S. Pattern* 



Nov. 14, 2000 



Sheet 12 of 24 



6,147,742 



LL 



< 
5 cc 
O <g 

o 



Q 
LU 



C/) 



UJ 

o 
cc 

O 

CO 

z 
o 

< 

z 

S3 
cc 
o 



o 
cc 

U- 



z 
3 m 



CO 

in 

HI 
CO 



> 
cc 
O 
O 

LU 
I- 
< 



= 9 
o - 1 



50 LU 



111 



X 

o 

< Q 

CD 



. Q 

UJ LU 
Q ^ 



Ui£ 



m 



CO 



CO 



10 



CO 



m 



in 



in 



CVJ 



in 



CO 

ai 

CO 

in 



o 
o 



CO 



CO 



in 
o 

CD 
CO 

o 



0 



m 



> 
< 



CO 
CM 

CO 



00 

CM 

>>• 

CO 



CM 
CM 
CO 



CO 

CO 
CM 

CO 



CO 
Q 

CC 

o 



UJ 
O 

z 
< 

CD 

< 



00 

CM 



CM 
CM 
O 
CM 

m 
00 

CM 

o 

CO 
CO 

<>* 
a> 

o 
in 

(D 
W 

Q_ 

-3 



CO 
CM 

o 

CVJ 

m 
"cm 

CO 
CO 
CO 

o 

o 
in 

O 

LU 
CL 



g> 
o 

in 
O 

UJ 
Q_ 



UJ 

o 
z 
< 

DO 



CO 
CM 



m 
cm 



CM 
CM 

o 

CM 

in 

CM 
cvi 



o 
25 

O 

UJ 
Q_ 



02/19/2004, EAST Version: 1.4.1 



U.S. Patent 



Nov. 14, 2000 



Sheet 13 of 24 



6,147,742 



CD 

5 



< 

2 CC 

LL < 

o 



Q 

LU 
LL 

O 
LU 
CL 

CO 

CC 
LU 
CO 
3 



LU 

o 
cc 

o 

CO 

z 
O 

< 
z 

52 
cc 
o 



o 
cc 



z 



CO 
LU 

UJ 
CO 



>- 
oc 
O 
O 

LU 
H 
< 
O 



= 9 



<z"o 
gco lu 



LU 



X 

o * 
< Q 

CO 



■ Q 

Q- LLJ 



LU LU 

<i 



lT> 



CvJ 



03 



00 



CO 



in 



in 



in 



CO 



in 



<Si 

CM 
O 
CM 

in 



CO 

cvi 

CVJ 

o 

C\J 

m 



CM 

cvi 

CM 

o 

CM 

m 



CM 
CM 

o 

CM 

»n 



CD 
O 

in 



CM 



CO 
O 

d 



CO 
CO 



CD 



CVJ 



CO 

o 

CO 
CO 



CO 

T— 

CO 
CO 

d 



CD 



m 

> 
< 



CO 

o 

CO 
CO 



CM 
CO 
CO 
CO 

d 



in 

> 
< 



CO 

o 

CO 
CO 



CO 
CO 
CO 

d 



in 

> 
< 

5 



CO 

o 

CO 
CO 



in 

CM 

d 
d 



m 

> 
< 



02/19/2004, EAST Version: 1.4.1 



U.S. Patent Nov. 14, 2000 Sheet 14 of 24 6,147, 



UTILIZING MERGE ID TO 
GATHER FILE INFORMATION 



SORTING IMAGES 
CHRONOLOGICALLY 



174 



176 



CALCULATING ELAPSED TIME 
BETWEEN IMAGE EXPOSURES 



IDENTIFYING NATURAL 
GROUPS OF IMAGES 



LOOKING FOR PROMOTE TO 
LEAD INDICATION 



178 



180 



182 



MARKING SERIES 
IMAGE SETS 




YES 



i 



BREAK-DOWN GROUPS 
INTO SUB-GROUPS 



CREATING VIEW-MENU 
PAGE 



184 



188 



190 



FIG. 12 



02/19/2004, EAST Version: 1.4.1 



U.S. Patent 



Nov. 14, 2000 



Sheet 15 of 24 



6,147,742 



< 

2 OC 
O w 

UL < 

o 



a 

UJ 
LL 

o 

UJ 

a. 

CO 

oc 

UJ 
CO 

3 



00 

g 

LL 



UJ 

o 

QC 

z> 
O 

CO 

o 

< 

QC 
O 

O 
QC 



o 



CO 
UJ 

UJ 

CO 



oc 

o 

CD 

UJ 

< 



o < 
oc y 



ceo UJ 



LU 

2s 



X 

o =* 

GO 



O.UJ 



LU UJ 

<? 

Q 



co 
o 

CD 

o 
o 

to 



CD 



in 



CO 



CM 



in 



m m 



in 



m 



CVJ 

^- 

CO 

in 



CVJ 



CO 

m 



CO 

cvi 
o 
in 



O 

UJ 
CL 
"3 



02/19/2004, EAST Version: 1.4.1 



U.S. Patent Nov. 14,2000 Sheet 16 of 24 6,147,742 



< 





1 IMAGE 


GROUP 
ID 








» C 


) ^ 




cc 


> cc 


> CO 






— 


OC 


► o- 


> cr. 


• 9 




SERIES? 


































FROM 
CAMERA 




AUDIO- 
IMAGE LINK 


134728:7 


134728:8 


134728:9 


134728:10 


134728:11 




134728:15 I 


134728:16 


134728:17 


134728:18 


134728:19 




134603:7 I 


571349:1 


571349:2 


134603:8 


USER SPECIFIED 




SERIES 
ID 




































CATEGORY 


OUR KIDS 


OUR KIDS 


OUR KIDS 


OUR KIDS 


AMBIANCE 


ACTIVITIES 


ACTIVITIES 


ACTIVITIES 


ACTIVITIES 


ACTIVITIES 




SCENERY 






OUR KIDS 




GROUP 
LEAD 


































DURA- 
TION, 
SEC 






























IOURCE 




FRAME 
ID # 


cr> 


o 

CO 


O) 
CM 


oo 

CVJ 


CM 


CO 
CM 


CM 
CM 


CM 


o 

CM 


O) 


CO 




CM 


m 


JATION S 




BATCH 
ID # 


572022 


572022 


572022 


572022 


572022 




572022 


572022 


572022 


572022 


572022 




572022 


571349 


571349 


572022 I 


FROM ORIGIh 




DATE/ CAP- 
TIME TURED 


5/10/97 6:51:30 


5/10/97 7:17:43 


5/10/97 8:30:10 


5/10/97 8:30:28 


15/10/97 10:33:32 




[5/10/97 14:20:49 


5/10/97 14:31:03 


5/10/97 14:40:43 


5/10/97 14:43:27 


5/10/97 15:10:12 




5/10/97 16:47:16 


5/11/97 10:32:44 


5/11/97 10:33:04 


5/11/97 10:33:32 | 






FILE 
TYPE 


JPEG 


CD < 
u 

L 

"3 


xi 

l, 

"3 


CD 

XI 

a. 

"3 


CD 

LU 

a. 
-> 




JPEG 


CD ( 

UJ 


CD i 

LU 

35 


'3 ( 

XI L 
1- t 
"5 ■ 


D 
u 




CD ( 

UJ 

a. 
-» 


CD ( 

UJ 1 

a. i 

-3 ' 


CD < 

XI 1 

a. i 

"3 - 


CD 

XI 

a. 

"3 



02/19/2004, EAST Version: 1.4.1 



U.S. Patent Nov. 14,2000 Sheet 17 of 24 6,147,742 



GROUP 
ID 




: cv 


I Cv 


c\ 


J c 




C£ 




- £ 




1 CD 


SERIES? 
























AUDIO- 
IMAGE LINK 


571349:3 


134603:9 


134603:10 


134603:11 


571349:4 




571349:20 I 


571349:21 


571349:22 


571349:23 


571349:24 


SERIES 
ID 
























CATEGORY 




OUR KIDS 


OUR KIDS 


OUR KIDS 
















GROUP 
LEAD 


>- 






















DURA- 
TION, 
SEC 
























FRAME 
ID # 


CO 




CO 


CVJ 


^- 




o 

CvJ 


CVJ 


CVJ 
CVJ 


CO 
CVJ 


CVJ 


BATCH 
ID # 


571349 


572022 


572022 


572022 


571349 




571349 


571349 


571349 


571349 


571349 


CAP- 
TURED 


10:36:13 


10:36:21 


CVJ 

CD 
CO 

o 


10:36:46 


10:39:20 




14:21:35 


14:30:37 


14:50:16 


14:55:29 


15:02:31 1 


DATE/ 
TIME 


5/11/97 


i 

CX> J 

3? I 


N 
J> 
r- 

r— 


o> 
/) 






5/11/97 


5/11/97 


i 

a> 
x> i 


i 

?) t 


o 


FILE 
TYPE 


JPEG 


3 ( 

U l 
1. C 
"3 " 


5 ( 

U l 

L I 


3 < 
II I 

L l 
■ 


3 

LU 
X 
"O 




(3 < 

LU I 

i. i 


3 ( 
U l 
X ( 

-? ■ 


3 < 
U I 
1. c 

"O ■ 


5 < 
U l 
X £ 

"3 ' 


3 
u 
l. 



CO 

5 



02/19/2004, EAST Version: 1.4.1 



UoSo Patent Nov. 14, 2000 Sheet 18 of 24 6 9 147 9 742 



Q 
yj 

LL 

O 
LU 
CL 
CO 

QC 
LU 
CO 

z> 



< 

LO 

5 



LU 

o 

QC 

o 

CO 

o 

< 

g 
o 



o 

cc 



co 

UJ _ 

LU 
CO 



> 
cc 

o 
o 

UJ 

< 

o 



b 9 
o < 



<zo 
pro lu 



UJ 
LL 



X 

O 
CD 



Q. LU 
<CC 

o2 



LU UJ 

< i— 

G 



CO 
Q 

cc 

ZD 

o 



CO 



> 


>- 




DC 


QC 


CC 


LU 


LU 


LU 








LU 


LU 


LU 


O 


O 


o 


CO 


CO 


CO 



00 



CO 



LO 



CO 
Q 



CO 
Q 

CC 

o 



CM 



CM 
CM 
O 
CM 

LO 

~co 

CD 
CO 

d 



LO 



co 
in 
o" 

CM 

o> 

CO 

d 



cn 



CD 



LO 

O 
LU 
Q- 



LO 
O 

LU 
CL 



LO 

O 
LU 
CL 
~3 



CM 

CM 
1^ 
LO 

"o 

CO 

d 
d 



LO 

O 
UJ 

CL 



CM 
< 
LO 



CD 



< 
LO 



< 

LO 

CD 
LL 



g 

LL 



02/19/2004, EAST Version: 1.4.1 



U.S. Patent 



Nov. 14, 2000 



Sheet 19 of 24 



6,147,742 



< 

LO 

5 



< 

O !g 



o- 1 



Q_ 
3 

OQ 



CM 



CO 



CO 



CO 
UJ 

DC 
LU 
CO 



z 



cb 

C\J 

x* 

CO 



CO 
CO 
CNJ 

co 



CO 
C\J 

CO 



CO 



CO 



CO 



CO 

o 

CO 
CO 



CO 

o 

CO 

co 



xf 

CO 

in 



CM 



CO 



CO 



CO 

o 

CO 
CO 



xt 
6> 
xr 

CO 



in 
o> 
xf 

CO 



CM 
CO 

o 

CO 
CO 



02/19/2004, EAST Version: 1.4.1 



U.S. Patent 



Nov. 14, 2000 



Sheet 20 of 24 



6,147,742 



GROUP 
LEAD 


> 








>■ 






























GROUP 
ID 


in 


m 


in 


LO 


CO 


CO 


CO 


CO 


CD 


CO 


CD 


CO 


CO 


CO 


CO 


CO 


CO 


CO 


co 


SERIES? 




>- 


>- 


>• 




> 


> 


> 
























AUDIO- 
IMAGE LINK 


571349:6 


571349:7 


571349:8 


571349:9 


571349:10 


571349:11 


571349:12 


571349:13 


571349:14 


571349:15 


571349:16 


571349:17 


571349:18 


571349:19 


571349:20 


571349:21 


571349:22 


571349:23 


571349:24 


SERIES 
ID 










>- 


>- 


> 


> 
























CATE- 
GORY 








































GROUP 
LEAD 








































DURA- 
TION, 
SEC 


















176.21 








305.39 














FRAME 
ID # 


CO 


r*. 


00 


O) 


o 

r— 


T— 


CM 
y— 


COi 




in 


co 




CO 


o> 


o 

CNJ 




CVJ 
CVJ 


co 

CVJ 


CVJ 


BATCH 
ID # 


571349 


571349 


571349 


571349 


571349 


571349 


571349 


571349 


571349 


571349 


571349 


571349 


571349 


571349 


571349 


571349 


571349 


571349 


571349 


DATE/ CAP- 
TIME TURED 


5/11/97 10:42:51 


5/11/97 Tfc09:04 


5/11/97 11:09:31 


5/11/97 11:09:49 


5/11/97 13:12:53 


5/11/97 13:15:42 


5/11/97 13:16:03 


5/11/97 13:16:07 


5/11/97 13:19:51 


5/11/97 13:22:53 


5/11/97 13:32:33 


5/11/97 13:35:17 


5/11/97 14:02:02 


5/11/97 14:13:40 


5/11/97 1421:35 


5/11/97 14:30:37 


5/11/97 14:50:16 


5/11/97 14:55:29 


5/11/97 15:02:31 


FILE 
TYPE 


JPEG 


JPEG 


JPEG 


JPEG 


JPEG 


JPEG 


JPEG 


JPEG 


MPEG 


JPEG 


JPEG 


JPEG 


MPEG 


JPEG 


JPEG 


JPEG 


JPEG 


JPEG 


| JPEG 



5 



02/19/2004, EAST Version: 1.4.1 



U.S. Patent 



Nov. 14, 2000 



Sheet 21 of 24 



6,147,742 




02/19/2004, EAST Version: 1.4.1 



U.S. Patent Nov. 14, 2000 Sheet 22 of 24 



UTILIZING MERGE ID TO 
GATHER INFORMATON 



ORGANIZING IMAGES 

I 

DIVIDING AUDIO RECORDING 
INTO AUDIO PHRASES 

J 

DETERMINING AUDIO 
PHRASE DURATIONS 

I ~ 

ASSIGNING IMAGE GROUPS 
TO AUDIO PHRASES 

CALCULATING DWELL TIME BY 
DIVIDING DURATION OF AUDIO 
PHRASE AMONG NUMBER 
OF IMAGES IN GROUP 

* — 

CHECKING FOR 
ROUND-OFF ERROR 

1 ~ 

STORING SORTED TABLE 



FIG. 17 



DETERMINING IMAGE OUTPUT 
PATH REQUESTED 

LOADING RULES 
FOR FORMATTING 



PROCESSING DATA 
FILES IN-TURN 



FIG. 18 



6,147,742 

192 
194 

196 
198 
200 
202 

206 
208 

210 
212 
214 



02/19/2004, EAST Version: 1.4.1 



U.S. Patent 



Nov. 14, 2000 



Sheet 23 of 24 



6,147,742 




02/19/2004, EAST Version: 1.4.1 



U.S. Patent Nov. 14,2000 Sheet 24 of 24 6,147,742 




FIG. I9B 



02/19/2004, EAST Version: 1.4.1 



6,147,742 

1 2 

PHOTOFINISHING SYSTEM AND METHOD described in U.S. Pat. No. 5,128,700 to Inoue. This photo 

FOR AUTOMATED ADVANCED SERVICES system includes a camera utilizing both film and a memory 

INCLUDING IMAGE AND ASSOCIATED card. The film captures images while the memory card 

AUDIO DATA PROCESSING records audio data. In practice, the two mediums are main- 

FIELD OF THE INVENTION 5 ta ' ned ' n tnc possession of the photographer who must avoid 

mixing audio with the wrong images. Photofinishing for this 

The invention relates to a system and method for photo- p noto system comprises conventional methods, 

finishing and more particularly an automatic photofinishing Following photofinishing, the prints are returned to the 

system and method for managing and processing audio data cuslomer> who then inserts lhe finisned prinls and tne dala 

and image data. ]0 memorv card mt0 a spec i a i playback device to view the print 

BACKGROUND OF THE INVENTION while hearing its audio. Thus, for this approach, no advanced 

The increased use of computers in many aspects of Photofinishing services are enabled or required Therefore, 

photography offers a pathway to deliver a higher level of lhe need exIS ' s for a Ptotofiniatang system and method for 

service for consumers. Many consumers often prefer to """waging and sequencing the audio data that is integrated 

capture pictures with conventional film-to-print photo 15 w.th lhe images on the same storage media. This B the case 

systems, while others prefer movie cameras, camcorders or for lma & es » nd audl ° J 01 ""? . reoorded on . [ llm ™ d c fcr 

modern digital cameras. New modes of utilizing images are '"nsmitted data streams of digital images with audio from 

becoming increasingly popular with varying forms of com- dl S lta ! lma S e source , s - Fu " h f x ' lhe needexists , for a P hot °- 

munication. Common utilization modes include distributing „ »°'shmg system and method for managing and sequencing 

e-mail with images and related audio on the World Wide 20 groups of orders for photofinishing semces that result in an 

Web, sharing images by electronic display (television), in r ,e f aled iraa 8 e and audl ° P r ° ducl u - ^ s y s , lem and method 

manipulating images electronically, and archiving images of ,he P resent ,nvenllon satlsfles lbese needs - 

for subsequent retrieval. SUMMARY OF THE INVENTION 

The image applications described above typically require M xh e photofinishing system and method of the present 
a consumer to expend substantial time to ensure proper invention enable the effective management of images and 
processing of the images. However, many consumers often associated data from a variety of input sources. Moreover, 
lack the time to fully explore and take advantage of the automatic preparation of customer orders for a variety of 
various image utilization opportunities available. Thus, in output media and formats [ s ^ ma de possible, 
spite of new options for processing images, consumers may 3Q To realize the advanta g es described above, in one form 
not get involved with such opportunities. A more automated the invemion comprises A photofinishing system for auto- 
means of processing consumer images is highly desirable to matically processing image and associated image data pur- 
relieve the time burden associated with image utilization and suaQt t0 customer output reqU ests. The system includes an 
management. or( j er mana g er operative to receive and control processing of 

Several proposals for photo systems including media 35 the output requests and at least one source of image related 

integral with the film for data recording have been disclosed, data corresponding to the output request. An input interface 

necessitating advanced photofinishing techniques. One receives the image related data from the source and converts 

proposal, by Bell et al. in U.S. Pat. No. 5,276,472 describes lne data to a digital data stream. A memory is included for 

film having an integral magnetic layer for storing additional storing the digital data stream. The system further includes 

data such as audio. The data is read magnetically during 40 a d a ta parser disposed in communication with the memory 

photofinishing and written to each print for subsequent t0 extrac t selected data streams according to the order 

playback when prints are viewed. manager and to reduce the data into respective image files 

Similar proposals to the Bell photo system described having respective groups of data fields. An output module is 

above are disclosed by Stoneham (U.S. Pat. No. 5,363,158), responsive to the order manager and is operative to produce 

Cocca (U.S. Pat. No. 5,363,157), Norris (U.S. Pat. No. 45 a photofinished output organized with respect to the data 

5,521,663), and Hawkins et al. (U.S. Pat. No. 5,389,989). fields. 

These patents describe cameras that record conventional In ano ther form, the invention comprises a photofinishing 

images as well as audio data. The cameras generally contain method for automatically processing image and associated 

an optical recording module that enables data, such as audio, d ata in a photofinishing system pursuant to customer output 

to be written as a latent image onto the film. This is in 50 reque sts. The method includes the steps of receiving batch 

addition to the normal capture of conventional images requests for a specified output; accumulating data relating to 

formed from ambient light passing through the camera's tne batch requests through an input interface; transforming 

lens. The optical recording module typically includes a the accumulated data into a digital data stream; interpreting 

column of LED's to expose digital data onto the film. The and classifying the data into digital images and digital data 

audio is recorded immediately adjacent to each image 55 f ie i ds; establishing correspondence between the digital 

captured, or buffered and written to the film following all i mage s and associated digital data fields; and organizing the 

image captures. corresponding digital images and associated digital data files 

In one advanced photofinishing technique for processing into the specified output. 

APS film, an APS camera may utilize APS IX magnetic data other features and advantages of the present invention 

tracks to detect when audio data has been captured. This 60 will be apparent from the following detailed description 

provides a photofinisher with audio to image correlation when read in conjunction with the accompanying drawings, 
information at processing. However, solutions to enable a 

photofinisher to process and manage film having images BRIEF DESCRIPTION OF THE DRAWINGS 

plus data, such as audio data, have not yet been adequately FIG. 1 is a block diagram of a photofinishing system 

described. 65 according to one embodiment of the present invention; 

Photo systems that integrate audio data separate from the FIG. 1A is a rear view of one input source according to the 

film have also been proposed. Such a two media system is present invention; 
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FIG. 2 is a block diagram of steps in a photofinishing downloaded. Moreover, information from video and audio 

method according to one embodiment of the present inven- cassettes 33 and 35 as well as audio CDS 37, photo CDS 39, 

tion; and picture discs 41 is envisioned. 

FIG. 3 is a block diagram of specific steps involved in the Referring now to FIG. 1A, a preferred input source 

method of FIG, 2* 5 es P e cially suitable for use in the present invention comprises 

a ■ ii i l » c c . • ij-i. a camera 43 including a housing 45 that includes a back 

FIG. 4 is a block diagram of specific steps involved in the , f „ ° u <r - i- A «n 

u a f urn 1 P ane * * or mountm g respective "Series Link and "Promote 

method ot MU 2; .to Lead" buttons 47 and 49. An LCD 51 disposed on the 

FIG. 5 is a schematic diagram of a strip of photographic back panel beneath a viewfinder 53 communicates the image 
film having latent image data; JQ number and audio status to the user. Mounted adjacent the 

FIG. 6 is a block diagram of specific steps involved in LCD is a microphone 55 for picking up audio signals related 
processing the latent image data of FIG. 5; t0 captured images. 

FIG. 7 is a graph of write element number versus film * ^ m di ^ tizer 27 >video/audio digitizer 28 and buffers29 
dcnsitv transform each set of data into a digital data stream. 

The output of the input interface feeds a digital data stream 
FIG. 8 is a look-up table for use with the steps of FIG. 6; « stQrage unit 34 where the da(a slfeam mformation ^ retained 

FIG. 9 is a graph of average film density sorted by in a mass memory at relatively high burst rates, 

increasing tone versus raw analog-digital code value; To create meaningful data files from the stored data 

FIG. 10 is a block diagram of specific steps utilized in the stream, the data parser 36 processes input data from the data 

method of FIG 2* stream storage unit 34. The parser breaks down the data into 

FIGS. 11 A and 11B are respective pages of a look-up table 2 ° a Pl»"Mty of fifc-types. establishes decoding calibration 

., , r , r £, T „ - „ r & r records, and creates interpreted digital image and audio files 

compiled from the steps of FIG. 10; in a digital data fi]e unit £ ^ ^ fik stQrage ^ 

FIG. 12 is a block diagram of specific steps utilized in the comprises a repository for formatted digital files while they 

method of FIG. 2; are being organized for output. 

FIG. 13 is a look-up table compiled from the steps of FIG. 25 The output interface 40 comprises a plurality of modules 

12; that organize image and, for example, audio data in a manner 

FIGS. 14A and 14B are respective pages of a look-up consistent with the requests received by the order manager 

table compiled from the steps of FIG. 12; 22 : ™ e f lc . rf ?? e 1 i ncludes a ° automatic arrange-it 

™™ , r unit 42 to begin the initial data compilation and organization 

FIGS. 15A and 15B are respective pages of a look-up 3Q t0 generate an orga nized image set. An automatic build-it 

table similar to FIGS. 14A and 14B; unit 44 is disposed at the output of the ar range-it unit to 

FIG. 16 is a view menu page created from the steps of receive the organized image set and complete the requisite 

FIG. 12; formatting and encoding for the specified output media. A 

FIG. 17 is a block diagram of specific steps utilized in the media writer 46 > such as a di S itaI film > P a P er or CD writer > 
method of FIG 2* 35 * s responsive to the build-it unit's formatting and encoding 

FIG. 18 is a block diagram of specific steps utilized in the l ° WritC lhe image data '° the SpCCifled ° UlpUl 

method of FIG. 2; and ' . 

r-i^r, , .^r, , r Operation of the photofinishing system of the present 

FIGS. 19A and 19B are representative output results from invcmion pr()ceeds according t0 steps carried out by each of 

the steps ot HO. 18. ^ (he uni(s described above> shown in FIGS 2 through 4, that 

DETAILED DESCRIPTION OF THE define the method of the present invention. 

INVENTION Referring now to FIG. 2, the method includes, generally, 

first receiving image batch requests with the order manager 

Referring now to FIG. 1, the photofinishing system of the 22, at step 50, accompanying rolls of film or electronic files, 

present invention, generally designated 20, provides an 45 The batch request may include a special code, hereafter 

automatic and integrated means to carry out photofinishing referred to as a merge code, to join separate batches of 

services. The system includes an order manager 22 for images and their data, such as audio data. The input interface 

controlling the processing of input information received by & employed, at step 52, to transform any non-digital image 

an input interface 30 to transform the multi-format data into and aud i G information into a common format digital data 

a common format digital data stream. A data parser 36 50 stream. The transforming step is followed by accumulating 

separates and classifies the various types of data for pack- d i g i ta l data through the input interface 30, such as image and 

aging through an output interface 40 pursuant to requests aud ; 0 information, at step 54, associated with the image 

received by the order manager. content. Additional information in the form of, for example 

Further referring to FIG. 1, the order manager 22 is audio data, may be within the roll of film or as part of the 

coupled to a variety of customer order sources including 55 electronic file. An interpretation and classification of the 

telecommunication networks linked to digital cameras, data, is carried out by the parser 36, at step 56 to properly 

remote kiosks, computers and scanners and operative to read break down or reduce the data. 

order envelopes associated with film rolls and other storage Following the step of interpreting the data, at step 56, the 

media, image and data content is then automatically gathered and 

The input interface 30 cooperates with the order manager 60 co-processed by the automatic arrange-it unit 42, at step 58, 

22 in receiving input image and audio data from a variety of for a specified customer or set of customers having submit- 

potential sources and in varying formats. Exemplary sources ted an identical merge code. A correspondence is 

include conventional photographic film 23, advanced photo established, at step 60, between digital images and digital 

services (APS) film, camera film with data on integral media data files, including audio data. Image and audio file sets are 

read by an IX media reader 25, and digital image and audio 65 then automatically sorted. Finally, they are formatted by the 

data from electronic still cameras (ESC) 26 or hybrid automatic build-it unit 44, at step 62, for a selected output 

cameras with audio data on separate media, and similarly path. 
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Specifically referring to FIGS. 2 and 3, the order receiving image data, but rather associated digital data. Abi-lcvel code 

step 50 (FIG. 2) carried out by the order manager 22 includes field B is written proximate to the start sentinel A and 

several sub-steps that define the overall functionality of the represents information specific to any particular image 

order manager. The order manager alternates between within the roll. The information may represent the cartridge 

checking for new incoming orders, at step 70, and managing 5 identification number, the number of audio recordings that 

the workflow of previously received orders among the follow, and so forth. A tone series field C is included to 

peripherals, at step 76. If a new order is received, the request enable photofinishing equipment to devise a transformation 

is catalogued, at step 72, for workload management. Such look " u P tab J e ' In fiIm s y stems > th j s to calibrate out 

cataloguing may include identifying the customer name, variations due to power supply fluctuations, Ught emitter 

address, services requested, job identification number, 10 a & m &> and tem P erature effects - 

merge code, image status and the like. Based on the services Additional data fields recorded on the film by the digital 

requested, the order manager compiles a workflow sequence film wnler ma y mclude a bi-level encoded start sentinel D, 

that is used to guide the overall process of advanced photo- for an individual data field that may include replicated 

finishing. Each step will be completed in sequence. A data calibration tones, associated image frame numbers, metrics 

file ID and input port ID is then relayed, at step 74, to the is representing the length of the audio recording, and the like, 

digital data stream storage unit 34 where data input rcceiv- Audio data content is conveyed by a binary coded digital 

ing ensues data stream fi^d E. This may be written as a 2 n tone series, 

„ . . , . .« * ... 4 . which may have a border line of regularly occurring Dmax 

Peripheral units such as the data parser will notify the . J . . . . .... . C1 * 

, v , 4 . . . cu iJ .u tones to assist with removing variability in the film transport 

order manager when they are idle, at step 76. Should the A , ~. . . ,.f. ~ . % t . . e \ t 

. a j • • ^ .u ?n speed and film position shifts. To signify the end of a data 

order manager find no new incoming orders, it then manages 20 c \ ... . r . , . . . . i c • i a 

. n 6 , . ... . \ ' . „ 7t _ . ° file, a bi-level encoded data start sentinel F is employed, 

the workload among the idle peripheral units. When idle, . , f *• • -i * a u rw .u 

. , . . i I • * i_ i_ c i . • which may also give information similar to field D for the 

peripherals are assigned their next job by first updating the °7 n - . t t^^c -i j c .t. 

r . \ 4 . t , , , t . J / -o , f r next audio recording. Referring now to FIGS. 2 and 6, the 

job status in the work order catalog, at step 78, and enabling . . jit- 7 /r^r- *\ • i j 

, t , 4 c . , . & . * ©a a j . interpreting and classifying step 56 (FIG. 2) includes a 

the subsequent photofinishing process, at step 80. A deter- * & , . "L i J ^ . * * » 

\ r , . . D -» i_ iL iL j n c parsing procedure that breaks down the data in the data 

mination is then made, at step 82, whether the order is 25 r to . r t , . 

. 4 , ,1- l . * * r .i_ • i_ • .1. stream into usable components or files. An exemplary pars- 
complete by checking the steps remaining for the job ui the * .. , ' . . . j . f ci 

j . i ir a ,u • • . j ■ ing procedure applied to the optical input data from film, 

order catalog. If no further processing is required, then the 6 , r t . , . . . j . j u j u -.u 

, . , & , .. , ' . , 4 & . n ' . such as the latent image data desenbed above with respect 

order has been delivered and the data is removed from the t 4 , . , & - . - x . t c 4 . 

t , qm • . . . . j to the accumulating step 54 (FIG. 2), involves first accessing 

catalog, at step 84, at which time the order manager 22 . t , , t . tU \u * . nft ^n. 

. & ! . 1 r . c , I j in the stored data stream with the parser 36, at step 120. The 

concludes its operations for that specific order. If the order 30 . , 4 . , t . 1 . j . u j 

, , ' . . « j i # i parser then determines whether an image data boundary is 

is incomplete, then the steps described above are repeated, n-iic L j -r j 

, . . 1 . , -a ., i • detected, at step 122. If a new boundary is found, a new 

beginning with step 70, until completion. . , ' C1 5 . , t t /. . , . , 

to to 1 1 image data file is created, at step 124, and the header tagged, 

Referring now to FIGS. 2 and 4, the transforming and at step 126 . filc image data fe then ent ered, at step 128 

accumulating steps, 52 and 54 (FIG. 2) include first receiv- until the end of the image boundary is detected, at step 130. 

ing notification by the order manager that a new data stream Qnce me end of ^ { bound&ry fa found> at step i 30 , 

awaits processing, then setting up a file identifier m the me cedure retums t0 the determination, at step 122, of 

digital data stream storage, collecting input data through the whether a new ^ daU fa detected Jf nQ { 

input interface 30 at step 94, from the input source, and boundary fa found) thc pareer 36 procecds by determining 

storing the collected data. ^ whether m audio data field start sen tinel is detected, at step 

Referring to FIG. 4, to continue the method of the present 132. If n o start sentinel is recognized, an inquiry is made 

invention within thc transforming step 52, a digitizer 27 or whether the detected data is the end of the data stream, at 

28 is employed to convert nondigital data formats to a digital step 134. jf ^ t h en the procedure stops, at step 136. If not, 

data stream. Of course, if the data source generates digital then the procedure loops back to step 122. If the start 

data, the transformation step is unnecessary. A file ID/locator 45 sentinel is detected at step 132, then a new audio data file is 

index is setup for the data stream, at step 96. A specified created, at step 140, and the file header is tagged with 

input port is then enabled, at step 98, which allows the data ensuing data such as the CID number or digital camera ID 

stream to be received and stored in the storage unit as a filc, number and so forth, at step 142. 

at step 100. Upon receipt of an end-of-file marker, the status pars i Dg operation then initiates a calibration process 

of the data file is then updated, at step 102, with the order 5Q that involves ^ calibrating individual write elements of 

manager 22. me Qj m data writer, at step 144. Each writer element 

If no data stream is in waiting, then the order manager 22 typically writes the same calibration tones, and as a result, 

determines whether the parser 36 is busy. If not, the order the digital values scanned from the film are theoretically 

manager enables the data parser to begin processing a identical. However, due to writer head manufacturing 

particular data stream. When, at step 104, a parse request has 55 variations, the values differ slightly. FIG. 7 comprises a 

been received by the storage unit, it then looks-up the index graph of the digital film density readings from each film 

by order file ID, at step 106, and relays the specified data writer element for a single tone that may require both gain 

stream to the parser, at step 108. If a parse request has not and offset corrections. The corrections may be determined 

been received, the storage unit notifies the order manager by using the median value for each written tone as the 

that it is idle and loops back to step 94. 60 reference value, then creating an error table by tone, for each 

An example of one of the more complex data formats writer element. A straightforward regression may be used to 

captured on film and capable of being efficiently processed derive the offset and gain correction for each writer element, 

by the present invention is shown schematically in FIG. 5. The resulting correction look-up table is depicted in FIG. 8. 

The film includes a plurality of data fields A, B, C, D, and Referring again to FIG. 6, following the step of calibrat- 

E, to robustly convey digital data as a latent image to a 65 ing the individual write elements at 144, a system calibration 

photofinisher. The first field A comprises a bi-level encoded is carried out, at step 146, to eliminate any variations due to 

data start sentinel that signifies that data following are not battery voltage, temperature and other influential effects. 
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The calibration ID scheme specifies the order in which each 
tone is written to film. Thus, a table may be created with two 
columns, one containing the ideal tonal values from the 
calibration ID specification, and the other containing the 
median digitized values from the previous step. FIG. 9 5 
depicts this type of data graphically. 

Once the calibration steps 144 and 146 are performed, the 
next operation involves rescaling the data, at 148, to span the 
full numerical range of potential output values. This may be 
carried out by satisfying the relation: 1Q 

Vi'=(Vi-Vmin+Vnew_rnin)(Vncw_max-VDCw_min)/(Vmax- 
Vmin) 

where: 

Vi=each raw data value 
Vi'=each rescaled data value 
Vmin«the smallest value in the raw data set 
Vmax«the largest value in the raw data set 
Vnew_max=the largest value of the resealed data; and 
Vnew_min=the smallest value of the rescaled data 
As the final part of the rescaling step 148, a regression curve 20 
fit is performed to transform the audio data back to the 
calibrated digital values. 

Further referring to FIG. 6, following the curve fit, the raw 
audio data is then converted to calibrated binary values and 
written to a data file, at step 150. Each raw data value is first 25 
corrected for the writer element variation by using the 
correction values for the write element that wrote that 
datum. This is readily accomplished by applying the gain 
and offset correction from the look-up table illustrated in 
FIG. 8, Next, the data is corrected for system variability, 30 
using the second transform relationship developed in step 
148. The regression equation derived from the tone scale 
calibration is then applied to data points to relate back to the 
actual original data value the camera intended to write. The 
converting step 150 continues until the audio data stop 35 
sentinel is detected, at step 152, at which time the procedure 
returns again to step 122. The parsing and calibration 
process is repeated until the data stream has been completely 
processed, reaching step 136. 

The parsing procedure for other forms of data, such as 40 
image data from film combined with data from other media, 
and image data from digital input sources includes steps 
similar to those steps described above relating to optical data 
from film. 

Referring now to FIGS. 2 and 10, following the interpret- 45 
ing and classifying procedure 56 (FIG. 2) carried out by the 
parser 36, the photofinishing method continues with the 
steps of automatically gathering and co-processing customer 
data and establishing a correspondence between digital 
images and digital data files, at steps 58 and 60 (FIG. 2) with 50 
the auto-arrange-it organizer 42. Generally, this involves 
determining the type of image output path requested, at step 
160 (FIG. 10), from the instruction of the order manager 22, 
and loading rules for organizing associated with the output 
path, at step 162. A merge identification code is then 55 
extracted, at step 164, which identifies all files to be included 
in the organization processing. A search is then carried out 
in the digital data file storage unit 34, at step 166, for files 
with the extracted merge ID code. Header content from all 
of the files retrieved from the search are compiled into a 60 
table, at step 168, with entries for each file to be included in 
the organization processing. Image-audio pairs are then 
linked, at step 170. They may be linked in an image centric 
scheme or an audio-centric scheme. To accomplish this, a 
number of sub-steps are necessary. Following organization 65 
of the table, the content is then passed, at step 172, to the 
auto-build-it module 44 (FIG. 1), 
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FIGS. 11A and 11B illustrate a correspondence table that 
might be constructed by the auto-arrange-it organizer 42 
according to the general steps above. Various information 
fields are provided for each file relating to data from the 
origination source, and the user. For each file type, such as 
a JPEG, MPEG or WAV format file, respective fields con- 
taining information such as date and time, batch ID #, frame 
ID #, audio snippet duration, are included. For processing an 
image having, for example, camera captured audio, a JPEG 
file image format, and associated with corresponding audio 
data having a WAV file format, a convenient image -audio 
link field is provided to cross-reference the files to each other 
and maintain the camera-specified correspondence through 
the photofinishing processing. 

With reference now to FIG. 12, after the correspondence 
table has been assembled, the information is then processed 
according to organizing rules loaded from the order manager 
22. The steps for an image-centric auto-arrange it photofin- 
ishing service for a CD-ROM output, according to one 
embodiment of the auto-arrange-it method, involves first 
using the customer merge ID, at step 174, to gather infor- 
mation about the submitted order content, including images 
and audio. The images are then sorted into chronological 
order, at step 176, to obtain the table shown in FIG. 13. The 
chronological sort keys to the date and time of the image 
capture to interleave all image batches successfully. 

Following the chronological sort, the elapsed time 
between image exposures is calculated, at step 178. This 
quantity is used to define each photographers normal time 
lapse pattern for the batch. The calculation may utilize a 
statistical measure to establish, for example, a standard 
deviation between picture to picture intervals. From this 
calculation, natural groups of images may be identified, at 
step 180, by photohabits and organized into an information 
table, such as that shown in FIGS. 14Aand 14B. Each group 
is given a sequential image group ID number for utilization 
later by the auto build-it module 44 (FIG, 1). 

The auto-arrange-it module 42 (FIG. 1) then looks within 
the identified groups for any "Promote to Group Lead" 
indication, at step 182 (FIG. 12). This information may be 
generated, for example, by the camera 43 having the select- 
able "Promote to Group Lead*' button 49. A table showing 
such information may be constructed, as shown in FIGS. 
ISAand 15B. This is a straightforward scan and resequenc- 
ing to move user-specified images out of chronological order 
to lead the natural group they are associated with. This step 
is particularly useful when the user wishes to have a 
CD-ROM created. The first image in each group generally 
serves as the visual navigation menu, so an image that best 
represents the group is ideal as the lead in each group. 
Further referring to FIG. 12, following the "Promote to 
Group Lead" determination, series image sets are marked, at 
step 184, by looking for a series link signal from a camera 
user or by noting sets of images with statistically short 
inter-picture intervals or by noting groups where image 
content has a strong data correlation. Series images are 
linked such that the playback delay time is reduced to create 
an effect of connectivity. FIGS. ISA and 15B illustrate a 
table showing the usersignaled and the automatic series tool. 
The column labeled "Series?" has a "Y" denoted in the table 
for images that were taken with an interval significantly 
shorter than the natural batch standard interval. For the 
examples listed in FIGS. 15A and 15B, the threshold for 
automatically connecting as a series was an interval under Va 
the standard deviation of the average inter-picture interval. 
For batch 572022, images taken at intervals under 2.44 
minutes were linked. For batch 571349, images taken at 
intervals under 8.04 minutes were linked. 
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Following the scries images marking step 184, a deter- 
mination is made whether the number of images within 
specific groups is too large, at step 186. If a group is too 
large, the group may be optionally broken, at step 188, into 
a number of sub-groups for quicker image location when 5 
visually searching. To navigate the CD contents, a view- 
menu page is created, at step 190, for the user. The page 
displays the pictures in groups, as shown in FIG. 16. The 
organization is then completed and the table is then stored in 
the digital data file storage unit 34 for subsequent CDROM 10 
burning. 

Alternatively, user specified categories may be utilized to 
organize the sequence and groupings of images. The general 
approach is similar to the image-centric case described 
above, but involves a category sort operation following the 15 
chronological sort step 176 (FIG. 12), Additionally, for high 
density media, such as a digital video disc (DVD), the image 
sorts can be saved in chronological groups and natural 
groups, as well as the user-specified categories. 

A further specific application for the auto-arrange-it mod- 20 
ule 42 involves audio-centric processing especially useful in 
the case of images with longer audio background 
soundtracks. Referring now to FIG. 17, the procedure begins 
by using the merge ID to gather file information, at step 192, 
for all of the submitted order content. The resulting table is 25 
similar to that described previously. The images are then 
organized, at step 194, as previously described in steps 176 
through 184 of FIG. 12. 

Following the image organizing step 194, the audio 
information is then organized. This involves first dividing 30 
each audio recording into audio phrases, at step 196, It is 
usually desirable to ensure that an image change will occur 
on a beat or at the end of a phrase. This may be done by 
analyzing the audio data versus time with an audio-oriented 
tool, many of which are MIDI -based and well known in the 35 
art. The durations of the audio phrases are then determined, 
at step 198. Each image group is then chronologically 
assigned to each corresponding chronological audio phrase, 
at step 200. Following the respective assignments, the dwell 
time for each image group within its audio phrase is 
calculated, at step 202 by dividing the total duration or play 
time of an audio phrase among the number of images in the 
group, taking into account the dwell time adjustment if 
images are denoted for series playback. The dwell times are 
then summed for the images in the group to check for 
round-off error, at step 206. The last image may be adjusted 
to match the end of the audio phrase, if necessary to 
complete the sorted table. The procedure concludes by 
storing, at step 208, the sorted table in the digital data file 
storage 38. 

The tables organized by the auto-arrange-it unit 42 are 
utilized by the auto-build-it unit 44 to process the output 
requested by the consumer. Referring now to FIG. 18, the 
steps performed by the unit include, generally, first deter- 
mining the type of image output path requested, at step 210 
from the information provided by the order manager. Rules 
are then loaded, at step 212, for formatting associated with 
the requested output path. The auto-build-it unit then 
accesses and utilizes the organization table, processing each 
specified data file in turn, at step 214. This may include 60 
creating header files, data files, intra-file linkage pointers 
and file to template linkages, dependent on the output 
desired. 

The general auto-build- it procedure described above is 
especially advantageous in producing collages of images, as 65 
illustrated in FIGS. 19A and 19B. Consistent with the steps 
outlined above, the auto-build-it module selects or creates a 



40 



45 



50 



55 



collage template with a number of image slots correspond- 
ing to the images in the customer order. The number of 
groups in the customer's image set may be used to specify 
how many large slots there arc in the template. The images 
are then linked to the template, with the lead image in each 
group assigned to a large slot and the subsequent images in 
each group assigned to the surrounding slots. After linking, 
each image is rescaled to the correct size for its assigned slot. 
Any customer requested title is added in and the order is then 
image processed to shape the tone scale and color gamut 
appropriate for hard copy or soft copy viewing. If the 
customer has requested to preview and approve the result 
before printing, the collage image is saved in the digital data 
file storage 38. The order manager 22 then e-mails an 
electronic copy to the customer at their home computer IP 
address or a neighborhood kiosk, as requested. 

If the output request comprises a variety of CD-ROM, the 
appropriate formatting is utilized to build the CD-ROM 
content. This formatting is well known in standards for 
multimedia CD-ROMS and DVDs. It's file structure usually 
includes an appropriate content directory and navigational 
instructions along with image files in PhotoCD, FlashPix or 
other format and audio files in AIFF, WAV or other format. 
Start-up application software is also usually included on the 
disk. 

Should the requested output comprise a traditional set of 
prints, chronologically arranged, the build-it module com- 
pletes the digital image processing required to convert the 
image from scanned negatives to printable densities that will 
drive a digital printer. This is also well known in the art. It 
typically involves the steps of inverting the image, adjusting 
the tone scale and color balance, and the like. 

If the output is a slide show for soft copy viewing or 
online photo albums, the images are similarly processed for 
soft copy display. The audio files are formatted according to 
computer-playback format requirements, following stan- 
dards formats such as AIFF or WAV. The order manager 22 
directs any intermediate output for user approval or modi- 
fication as well as the final output and delivery of the 
customer order. It manages the interaction with the billing 
system subsequently and releases disk space in the digital 
data stream storage 34 and the digital data file storage 38 
once orders have been completed. 

Those skilled in the art will appreciate the many benefits 
and advantages offered by the present invention. One impor- 
tant advantage involves the capability of managing and 
sequencing audio data integrated with images in a photo- 
finishing system and method. Additionally, the present 
invention provides the feature of managing and sequencing 
groups of orders for photofinishing services that result in an 
integrated image and audio product. 

While the invention has been particularly shown and 
described with reference to the preferred embodiments 
thereof, it will be understood by those skilled in the art that 
various changes in form and detail may be made therein 
without departing from the spirit and scope of the invention. 

PARTS LIST 

20 Photofinishing system 

22 Order manager 

23 Conventional photographic film 

25 IX media reader 

26 Electronic still camera 

27 Film digitizer 

28 Video/Audio digitizer 

29 Buffer 

30 Input interface 
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31 Buffer 

33 Video cassette 

34 Data stream storage unit 

35 Audio cassette 

36 Data parser 

37 Audio CD 

38 Digital data file storage unit 

39 Photo CD 

40 Output interface 

41 Picture disc 

42 Automatic arrange-it unit 

43 Camera 

44 Automatic build-it unit 

45 Housing 

46 Media writer 

47 Series link button 

49 Promote to lead button 

50 Receiving step 

51 LCD 

52 Transforming step 

53 Viewfinder 

54 Accumulating step 

55 Microphone 

56 Interpreting and classifying step 

58 Automatic gathering and co-processing step 

60 Establishing correspondence step 

62 Formatting step 

70 Checking step 

72 Cataloguing step 

74 Relaying step 

76 Managing step 

78 Updating step 

80 Enabling step 

82 Determining step 

84 Removing step 

94 Collecting step 

96 Setting-up step 

98 Enabling step 

100 Receiving step 

102 Updating step 

104 Receiving step 

106 Looking-up step 

108 Relaying step 

120 Accessing step 

122 Determining step 

124 Creating step 

126 Tagging step 

128 Entering step 

130 Detecting step 

132 Determining step 

134 Inquiring step 

136 Stopping step 

140 Creating step 

142 Tagging step 

144 Initiating step 

146 Calibrating step 

148 Rescaling step 

150 Converting step 

152 Detecting step 

160 Determining step 

162 Loading step 

164 Extracting step 

166 Searching step 

168 Compiling step 

170 Linking step 

172 Passing step 

174 Using step 
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176 Sorting step 

178 Calculating step 

180 Identifying step 

182 Looking step 
5 184 Marking step 

186 Determining step 

188 Breaking step 

190 Creating step 

192 Using step 
to 194 Organizing step 

196 Dividing step 

198 Determining step 

200 Assigning step 

202 Calculating step 
15 206 Summing step 

208 Storing step 

210 Determining step 

212 Loading step 

214 Processing step 
20 A First data field 

B Bi -level code field 

C Tone series field 

D Bi-level encoded start sentinel 

E Digital data stream field 
25 F Data start sentinel 
What is claimed is: 

1. A photo finishing system for automatically processing 
images from image related data in terms of image content 
and the content of non-image data associated with an image 

30 pursuant to customer output requests, said system including: 

an order manager operative to receive and control pro- 
cessing of said output requests; 

at least one source of image related data corresponding to 
said output request; 

an input interface for receiving said image related data 
from said at least one source and converting said data 
to a digital data stream; 

a memory for storing said image related data; 
40 a data parser disposed in communication with said 
memory to extract selected data from said data stream 
according to said order manager and assemble said 
extracted data into respective files corresponding to 
image data and non- image data each having respective 
45 groups of data fields; and 

an output module responsive to said order manager and 
operative to produce an output organized with respect 
to said data fields. 

2. A photofinishing system according to claim 1 wherein 
50 said at least one source of image related data includes: 

photographic film; 

a film developer for processing said film into image 
negatives; and 

55 a film digitizer to convert said respective image negatives 
into digital data. 

3. A photofinishing system according to claim 2 wherein 
said photographic film comprises hybrid film having audio 
image information written on said film, said at least one 

60 source of image related data further including an IX reader. 

4. A photofinishing system according to claim 1 wherein 
said at least one source of image related data includes: 

electronic image data. 

5. A photofinishing system according to claim 1 wherein 
65 said at least one source of image related data includes: 

at least one input from the group comprising a video 
cassette tape and an audio cassette tape; and 
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a digitizer for converting said at least one input to digital 16. A method of processing latent image information 

form. according to claim 15 wherein said calibrating step includes 

6. A photofinishing system according to claim 1 wherein the step of: 

said at least one source of image related data includes: correcting for said respective write element gain and 

at least one digital input from the group comprising an 5 offset parameters 

audio CD and a photo CD and a picture disc. 17. A method of processing latent image information 

7. A photofinishing system according to claim 1 wherein according t0 claim 16 wherein said correcting step includes 
said output module includes: me steps 0 f. 

an auto-arrange it unit responsive to said data parser to . , c ... , . c 

organize said extracted data fields into tables 30 determiain S raedian values for said tonal information; 

8. A photofinishing system according to claim 7 wherein creating an error table for said tonal information; and 
said output module further includes: deriving said offset and gain correction from said table. 

an auto-build-it unit operative to package said image and 18. A method of processing latent image information 

associated audio data according to said tables. according to claim 15 wherein said calibrating step includes 

9. A photofinishing method for automatically processing J5 the step of: 

image data and associated data in a photofinishing system correcting for said write element variations, 

pursuant to customer output requests, said method including 19. A met hod of processing latent image information 

the steps of: according to claim 18 wherein said correcting step includes 

receiving a batch of requests, each request being for a the steps of: 

specified similar output; ^ creating a table of ideal tonal values and median digitized 

accumulating data relating to each of said batch of values- and 

requests through an input interface; derjving ' said aKKCted operating p arame t e r values from 

interpreting and classifying said accumulated data into sa j d la ^j e 

digital images and digital data fields each associated 20 A method of aut0 matically organizing image content 

with an image; 25 acc0 rding to an order for a requested image output path and 

establishing correspondence between said digital images a photofinishing service style, said method including the 

and digital data fields; and steps of: 

organizing said corresponding digital images and digital determining the type of requested image output path 

data fields into said specified output. requested; 

10. A photofinishing method according to claim 9 and 30 loading rules for organizing associated with said 
further including the step of: requested output path; 

transforming said accumulated data into a digital data gathering images identified with said photofinishing ser- 



stream. 



vice style request; 



11. Apholofinishingmethodaccordingtoclaim9wherein Vmk{n data with said imagcs t0 form an 
said organizing step includes the step of: 35 organiz ed content table; and 

automatically gathering anc I co-processing customer data. processing said organ i ze d content table to develop said 

12. A photofinishing method according to claim 9 wherein requested image output path. 

said organizing step includes the step of: 21 A method of automatically organizing image content 

identifying a merge code associated with respective sepa- according to claim 20 wherein said step of processing 

rate orders of said batch of orders; and 40 includes the step of . 

automatically gathering and co-processing customer data automatically gathering and co-processing customer data. 

from separate orders according to said merge code. 22 A method of automatically organizing image content 

13. A photofinishing method according to claim 9 wherein accord i ng t0 claim 20 wherein said linking step includes the 
said organizing step includes the step of: steps of 

automatically sorting and formatting image and audio file 45 identifying a merge code associated with respective sepa- 

sets from said digital data stream. rate orders . and 

14. A photofinishing method according to claim 9 wherein automatically gathering and co-processing customer data 
said organizing step includes the step of: ff0m 0fders acc0l6mg t0 said merge code _ 

automatically establishing correspondence between mul- ^ 23 A method of automatically organizing image content 

tiple data streams and origins. according to claim 20 wherein said processing step includes 

15. A method of processing latent image tonal information t he step of* 
recorded on photographic film by one or more write automatically sorting and formatting image and audio file 



elements, said method including the steps of: 



sets. 



developing said film to generate respective image infor- 55 24 A method of aul0 matically organizing image content 

rnatl0n; according to claim 20 wherein said processing step includes 

transforming said image information into a digital data t he step of: 

stream; automatically establishing correspondence between mul- 

parsing said information into respective parameter data t j p i e data streams and origins. 

recorded by said respective write elements; 60 25 . A method of automatically organizing image content 

calibrating said respective write elements to compensate according to claim 20 wherein said processing step includes 

for variations; the step of: 

resealing said data to span a predetermined full numerical organizing image content based on user-specified catego- 

range of potential output values; and ries. 

fitting said data along a predetermined regression function 65 26. A method of automatically organizing image content 

to transform said information back to calibrated digital according to claim 20 wherein said processing step includes 

values. the step of: 
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organizing image content based on the chronology of 
picture taking and statistics of inter-picture intervals. 

27. A method of automatically organizing image content 
according to claim 20 wherein said processing step includes 
the step of: 

organizing image content based on devising natural 
groups of pictures from inter-picture interval statistics. 

28. A method of automatically organizing image content 
according to claim 20 wherein said processing step includes 
the step of: 

organizing image content based on modifying image 
playback duration based on a perceived series. 

29. A method of automatically organizing image content 
according to claim 20 wherein said processing step includes 
the step of: 



i7,742 

16 

organizing image content based on overriding the prede- 
termined picture sequence when a promote to lead 
request is detected. 

30. A method of automatically organizing image content 
5 according to claim 20 wherein said loading rules step 

includes the step of: 

implementing audio-centric procedures for said process- 
ing step. 

31. A method of automatically organizing image content 
10 according to claim 20 wherein said loading rules step 

includes the step of: 

implementing image-centric procedures for said process- 
ing step. 

* * * * * 
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